Foundations of Emergency Medicine: Impact of a Standardized, Open-access, Core Content Curriculum on In-Training Exam Scores.

Introduction: Learners frequently benefit from modalities such as small-group, case-based teaching and interactive didactic experiences rather than passive learning methods. These contemporary techniques are features of Foundations of Emergency Medicine (FoEM) curricula, and particularly the Foundations I (F1) course, which targets first-year resident (PGY-1) learners. The American Board of Emergency Medicine administers the in-training exam (ITE) that provides an annual assessment of EM-specific medical knowledge. We sought to assess the effect of F1 implementation on ITE scores. Methods: We retrospectively analyzed data from interns at four EM residency programs accredited by the Accreditation Council for Graduate Medical Education. We collected data in 2021. Participating sites were geographically diverse and included three- and four-year training formats. We collected data from interns two years before (control group) and two years after (intervention group) implementation of F1 at each site. Year of F1 implementation ranged from 2015-2018 at participating sites. We abstracted data using a standard form including program, ITE raw score, year of ITE administration, US Medical Licensing Exam Step 1 score, Step 2 Clinical Knowledge (CK) score, and gender. We performed univariable and multivariable linear regression to explore differences between intervention and control groups. Results: We collected data for 180 PGY-1s. Step 1 and Step 2 CK scores were significant predictors of ITE in univariable analyses (both with P < 0.001). After accounting for Step 1 and Step 2 CK scores, we did not find F1 implementation to be a significant predictor of ITE score, P = 0.83. Conclusion: Implementation of F1 curricula did not show significant changes in performance on the ITE after controlling for important variables.

Reflections on augmented reality codes for teaching fundamental defensive techniques to boxing beginners.

AR technology allows users to interact with virtual objects in real-world settings. Immersive AR experiences can enhance creativity and possibilities. Learners can explore real-life situations in a safe, controlled environment, understand abstract concepts and solve problems. This study investigates whether AR-codes affect boxing beginners' performance in some fundamental defensive techniques. An experimental and control group were created to implement a quasi-experimental design. By using the ASSURE instructional design model, AR technology was incorporated into the educational program and delivered in flipped classroom method to the experimental group. Comparatively, the control group is taught a program using a teacher's command style. A post-measurement of defensive boxing skills was conducted for both groups. Participants were 60 boxing beginners aged 12 to 14 who had enrolled in Port Fouad Sports Club's 2023/2024 training season in Port Said, Egypt. Randomly, participants were divided into control and experimental groups. They were homogenized and equivalent in terms of age, height, weight, IQ, physical fitness, and skill level. According to the study results, the experimental group performed better in post-measurements than the control group. The AR Codes technology had a large effect size on the learning of boxing defensive skills under study. Consequently, it is necessary to use AR Codes technology as an educational resource to enhance the learning process, integrate it with active learning strategies, and use it to teach defensive boxing skills and apply them to offensive and counterattack skills, thereby improving the learning process.

Evaluation of the creation of podcasts for instructional delivery in a post-graduate training elective course.

OBJECTIVES: To describe the creation of podcasts for instructional delivery and evaluate strengths and areas for improvement in a post-graduate training (PGT) elective course. METHODS: After creating a podcast series, students in the PGT elective from Spring 2021 to Fall 2022 listened to the series then completed a reflection based on five open-ended questions that provoked their thoughts and feelings about use of podcasts as a method of delivering information and teaching. Responses were downloaded and a content analysis was performed. Each investigator analyzed responses from all reflections to identify major themes and subthemes. Letter of intent assignment and overall course grades were compared to assess if podcasts affected student learning. RESULTS: Ninety-one students provided reflections about the use of podcasts in the PGT elective course, which revealed three major themes with 13 subthemes, including perception of guests, learner experience, and show and episode production. Students appreciated the various perspectives, authenticity, relatability, and diversity of the guest speakers; the learning environment was described as flexible, relatable, positive, and a safe space; the podcast design was noted to be informative, organized, and easily accessible. Areas for improvement included more interaction with guests and more visuals. Letter of intent assignment and overall course grades were similar before and after podcast implementation. CONCLUSION: The use of podcasts as an educational tool in a PGT elective course had a variety of characteristics that students preferred to traditional lecture-style classes.

Improving the ability to discriminate medical multiple-choice questions through the analysis of the competitive examination to assign residency positions in Spain.

INTRODUCTION: Psychometrics plays a vital role in evaluating educational research, including the analysis of multiple-choice exams. This study aims to improve the discriminatory ability of the "Médico Interno Residente" (MIR) medical exam in Spain, used to rank candidates for specialized healthcare training, through psychometric analysis. METHODS: We analyzed 2,890 MIR exam questions from 2009 to 2021 (totaling 147,214 exams), categorizing them based on methodology and response type. Evaluation employed classical test theory and item response theory (IRT). Classical test theory determined difficulty and discrimination indices, while IRT assessed the relationship between knowledge levels and question performance. RESULTS: Question distribution varied across categories and years. Frequently addressed knowledge areas included various medical specialties. Non-image-associated clinical cases were the easiest, while case-based clinical questions exhibited the highest discriminatory capacity, differing significantly from image-based case or negative questions. High-quality questions without images had longer stems but shorter answer choices. Adding images reduced discriminatory power and question difficulty, with image-based questions being easier. Clinical cases with images had shorter stems and longer answer choices. CONCLUSIONS: For improved exam performance, we recommend using a clinical case format followed by direct short-answer questions. Questions should be of low difficulty, providing clear and specific answers based on scientific evidence and avoiding ambiguity. Typical clinical cases with key characteristic features should be presented, excluding uncertain boundaries of medical knowledge. Questions should have lengthy stems and concise answer choices, minimizing speculation. If images are used, they should be typical, clear, consistent with the exam, and presented within clinical cases using clinical semiotics and propaedeutics.

Performance of ChatGPT on the Taiwan urology board examination: insights into current strengths and shortcomings.

PURPOSE: To compare ChatGPT-4 and ChatGPT-3.5's performance on Taiwan urology board examination (TUBE), focusing on answer accuracy, explanation consistency, and uncertainty management tactics to minimize score penalties from incorrect responses across 12 urology domains. METHODS: 450 multiple-choice questions from TUBE(2020-2022) were presented to two models. Three urologists assessed correctness and consistency of each response. Accuracy quantifies correct answers; consistency assesses logic and coherence in explanations out of total responses, alongside a penalty reduction experiment with prompt variations. Univariate logistic regression was applied for subgroup comparison. RESULTS: ChatGPT-4 showed strengths in urology, achieved an overall accuracy of 57.8%, with annual accuracies of 64.7% (2020), 58.0% (2021), and 50.7% (2022), significantly surpassing ChatGPT-3.5 (33.8%, OR = 2.68, 95% CI [2.05-3.52]). It could have passed the TUBE written exams if solely based on accuracy but failed in the final score due to penalties. ChatGPT-4 displayed a declining accuracy trend over time. Variability in accuracy across 12 urological domains was noted, with more frequently updated knowledge domains showing lower accuracy (53.2% vs. 62.2%, OR = 0.69, p = 0.05). A high consistency rate of 91.6% in explanations across all domains indicates reliable delivery of coherent and logical information. The simple prompt outperformed strategy-based prompts in accuracy (60% vs. 40%, p = 0.016), highlighting ChatGPT's limitations in its inability to accurately self-assess uncertainty and a tendency towards overconfidence, which may hinder medical decision-making. CONCLUSIONS: ChatGPT-4's high accuracy and consistent explanations in urology board examination demonstrate its potential in medical information processing. However, its limitations in self-assessment and overconfidence necessitate caution in its application, especially for inexperienced users. These insights call for ongoing advancements of urology-specific AI tools.

Learning with AI Language Models: Guidelines for the Development and Scoring of Medical Questions for Higher Education.

In medical and biomedical education, traditional teaching methods often struggle to engage students and promote critical thinking. The use of AI language models has the potential to transform teaching and learning practices by offering an innovative, active learning approach that promotes intellectual curiosity and deeper understanding. To effectively integrate AI language models into biomedical education, it is essential for educators to understand the benefits and limitations of these tools and how they can be employed to achieve high-level learning outcomes.This article explores the use of AI language models in biomedical education, focusing on their application in both classroom teaching and learning assignments. Using the SOLO taxonomy as a framework, I discuss strategies for designing questions that challenge students to exercise critical thinking and problem-solving skills, even when assisted by AI models. Additionally, I propose a scoring rubric for evaluating student performance when collaborating with AI language models, ensuring a comprehensive assessment of their learning outcomes.AI language models offer a promising opportunity for enhancing student engagement and promoting active learning in the biomedical field. Understanding the potential use of these technologies allows educators to create learning experiences that are fit for their students' needs, encouraging intellectual curiosity and a deeper understanding of complex subjects. The application of these tools will be fundamental to provide more effective and engaging learning experiences for students in the future.

R.I.M.E. and reason: multi-station OSCE enhancement to neutralize grade inflation.

To offset grade inflation, many clerkships combine faculty evaluations with objective assessments including the Medical Examiners Subject Examination (NBME-SE) or Objective Structured Clinical Examination (OSCE), however, standardized methods are not established. Following a curriculum transition removing faculty clinical evaluations from summative grading, final clerkship designations of fail (F), pass (P), and pass-with-distinction (PD) were determined by combined NBME-SE and OSCE performance, with overall PD for the clerkship requiring meeting this threshold in both. At the time, 90% of students achieved PD on the Internal Medicine (IM) OSCE resulting in overall clerkship grades primarily determined by the NBME-SE. The clerkship sought to enhance the OSCE to provide a more thorough objective clinical skills assessment, offset grade inflation, and reduce the NBME-SE primary determination of the final clerkship grade. The single-station 43-point OSCE was enhanced to a three-station 75-point OSCE using the Reporter-Interpreter-Manager-Educator (RIME) framework to align patient encounters with targeted assessments of progressive skills and competencies related to the clerkship rotation. Student performances were evaluated pre- and post-OSCE enhancement. Student surveys provided feedback about the clinical realism of the OSCE and the difficulty. Pre-intervention OSCE scores were more tightly clustered (SD = 5.65%) around a high average performance with scores being highly negatively skewed. Post-intervention OSCE scores were more dispersed (SD = 6.88%) around a lower average with scores being far less skewed resulting in an approximately normal distribution. This lowered the total number of students achieving PD on the OSCE and PD in the clerkship, thus reducing the relative weight of the NMBE-SE in the overall clerkship grade. Student response was positive, indicating the examination was fair and reflective of their clinical experiences. Through structured development, OSCE assessment can provide a realistic and objective measurement of clinical performance as part of the summative evaluation of students.

Improved cardiac auscultation competency interweaving visual, auditory, and tactile stimuli: a preliminary study.

Methods:   A pilot randomized controlled trial was conducted at our institution's simulation center with 32 first year medical students from a single medical institution. Participants were randomly divided into two equal groups and completed an educational module the identification and pathophysiology of five common cardiac sounds. The control group utilized traditional education methods, while the interventional group incorporated multisensory stimuli. Afterwards, participants listened to randomly selected cardiac sounds and competency data was collected through a multiple-choice post-assessment in both groups. Mann-Whitney U test was used to analyze the data. Results: Data were analyzed using the Mann-Whitney U test. Diagnostic accuracy was significantly higher in the multisensory group (Mdn=100%) compared to the control group (Mdn=60%) on the post-assessment (U=73.5, p<0.042). Likewise, knowledge acquisition was substantially better in the multisensory group (Mdn=80%) than in the control group (Mdn=50%) (U= 49, p<0.031). Conclusions: These findings suggest the incorporation of multisensory stimuli significantly improves cardiac auscultation competency. Given its cost-effectiveness and simplicity, this approach offers a viable alternative to more expensive simulation technologies like the Harvey simulator, particularly in settings with limited resources. Consequently, this teaching modality holds promise for global applicability, addressing the worldwide deterioration in cardiac auscultation skills and potentially leading to better patient outcomes. Future studies should broaden the sample size, span multiple institutions, and investigate long-term retention rates.

Utility of RAND/UCLA appropriateness method in validating multiple-choice questions on ECG.

OBJECTIVES: This study aimed to investigate the utility of the RAND/UCLA appropriateness method (RAM) in validating expert consensus-based multiple-choice questions (MCQs) on electrocardiogram (ECG). METHODS: According to the RAM user's manual, nine panelists comprising various experts who routinely handle ECGs were asked to reach a consensus in three phases: a preparatory phase (round 0), an online test phase (round 1), and a face-to-face expert panel meeting (round 2). In round 0, the objectives and future timeline of the study were elucidated to the nine expert panelists with a summary of relevant literature. In round 1, 100 ECG questions prepared by two skilled cardiologists were answered, and the success rate was calculated by dividing the number of correct answers by 9. Furthermore, the questions were stratified into "Appropriate," "Discussion," or "Inappropriate" according to the median score and interquartile range (IQR) of appropriateness rating by nine panelists. In round 2, the validity of the 100 ECG questions was discussed in an expert panel meeting according to the results of round 1 and finally reassessed as "Appropriate," "Candidate," "Revision," and "Defer." RESULTS: In round 1 results, the average success rate of the nine experts was 0.89. Using the median score and IQR, 54 questions were classified as " Discussion." In the expert panel meeting in round 2, 23% of the original 100 questions was ultimately deemed inappropriate, although they had been prepared by two skilled cardiologists. Most of the 46 questions categorized as "Appropriate" using the median score and IQR in round 1 were considered "Appropriate" even after round 2 (44/46, 95.7%). CONCLUSIONS: The use of the median score and IQR allowed for a more objective determination of question validity. The RAM may help select appropriate questions, contributing to the preparation of higher-quality tests.

Item analysis: the impact of distractor efficiency on the difficulty index and discrimination power of multiple-choice items.

BACKGROUND: Distractor efficiency (DE) of multiple-choice questions (MCQs) responses is a component of the psychometric analysis used by the examiners to evaluate the distractors' credibility and functionality. This study was conducted to evaluate the impact of the DE on the difficulty and discrimination indices. METHODS: This cross-sectional study was conducted from April to June 2023. It utilizes the final exam of the Principles of Diseases Course with 45 s-year students. The exam consisted of 60 type A MCQs. Item analysis (IA) was generated to evaluate KR20, difficulty index (DIF), discrimination index (DIS), and distractor efficiency (DE). DIF was calculated as the percentage of examinees who scored the item correctly. DIS is an item's ability to discriminate between higher and lower 27% of examinees. For DE, any distractor selected by less than 5% is considered nonfunctional, and items were classified according to the non-functional distractors. The correlation and significance of variance between DIF, DI, and DE were evaluated. RESULTS: The total number of examinees was 45. The KR-20 of the exam was 0.91. The mean (M), and standard deviation (SD) of the DIF of the exam was 37.5(19.1), and the majority (69.5%) were of acceptable difficulty. The M (SD) of the DIS was 0.46 (0.22), which is excellent. Most items were excellent in discrimination (69.5%), only two were not discriminating (13.6%), and the rest were of acceptable power (16.9%). Items with excellent and good efficiency represent 37.3% each, while only 3.4% were of poor efficiency. The correlation between DE and DIF (p = 0.000, r= -0.548) indicates that items with efficient distractors (low number of NFD) are associated with those having a low difficulty index (difficult items) and vice versa. The correlation between DE and DIS is significantly negative (P = 0.0476, r=-0.259). In such a correlation, items with efficient distractors are associated with low-discriminating items. CONCLUSIONS: There is a significant moderate negative correlation between DE and DIF (P = 0.00, r = -0.548) and a significant weak negative correlation between DE and DIS (P = 0.0476, r = -0.259). DIF has a non-significant negative correlation with DIS (P = 0.7124, r = -0.0492). DE impacts both DIF and DIS. Items with efficient distractors (low number of NFD) are associated with those having a low difficulty index (difficult items) and discriminating items. Improving the quality of DE will decrease the number of NFDs and result in items with acceptable levels of difficulty index and discrimination power.

Assessing ChatGPT 4.0's test performance and clinical diagnostic accuracy on USMLE STEP 2 CK and clinical case reports.

While there is data assessing the test performance of artificial intelligence (AI) chatbots, including the Generative Pre-trained Transformer 4.0 (GPT 4) chatbot (ChatGPT 4.0), there is scarce data on its diagnostic accuracy of clinical cases. We assessed the large language model (LLM), ChatGPT 4.0, on its ability to answer questions from the United States Medical Licensing Exam (USMLE) Step 2, as well as its ability to generate a differential diagnosis based on corresponding clinical vignettes from published case reports. A total of 109 Step 2 Clinical Knowledge (CK) practice questions were inputted into both ChatGPT 3.5 and ChatGPT 4.0, asking ChatGPT to pick the correct answer. Compared to its previous version, ChatGPT 3.5, we found improved accuracy of ChatGPT 4.0 when answering these questions, from 47.7 to 87.2% (p = 0.035) respectively. Utilizing the topics tested on Step 2 CK questions, we additionally found 63 corresponding published case report vignettes and asked ChatGPT 4.0 to come up with its top three differential diagnosis. ChatGPT 4.0 accurately created a shortlist of differential diagnoses in 74.6% of the 63 case reports (74.6%). We analyzed ChatGPT 4.0's confidence in its diagnosis by asking it to rank its top three differentials from most to least likely. Out of the 47 correct diagnoses, 33 were the first (70.2%) on the differential diagnosis list, 11 were second (23.4%), and three were third (6.4%). Our study shows the continued iterative improvement in ChatGPT's ability to answer standardized USMLE questions accurately and provides insights into ChatGPT's clinical diagnostic accuracy.

The effect of interactive learning method on nursing students' learning of movement requirement: A randomized controlled study.

BACKGROUND: Interactive learning is one of the active learning methods used to develop students' knowledge and skills. OBJECTIVES: This study was conducted to determine the effect of the interactive learning method with questions prepared according to Bloom's taxonomy on nursing students' learning of the need for movement. DESIGN: It is a randomized controlled study conducted in May 2023. PARTICIPANTS: This study was conducted with 134 first-year nursing students taking the Fundamentals of Nursing course. The students were randomly divided into control (n = 67) and experimental (n = 67). METHOD: The data was collected by using a form delineating descriptive characteristics, a test assessing the necessity of movement knowledge, an analysis of a specific case study, and an assessment form for gathering student opinions on interactive learning. The queries within the data collection form were tailored to align with Bloom's taxonomy. Following the theoretical instruction, each student participated in the mobility needs knowledge test and case analysis (pre-test). The instructional session for the control group involved conventional lecture-style teaching supplemented with a question-and-answer format, while the experimental group received instruction through an interactive learning approach. One week subsequent to this, all students retook the mobility needs knowledge test and case analysis (post-test). Subsequently, feedback regarding the interactive learning method was solicited from the students. RESULTS: The knowledge test revealed a statistically significant difference, with the control group exhibiting a higher median comprehension score in the post-test compared to the experimental group (p < 0.05). Regarding the case analysis, statistical analysis demonstrated that the experimental group surpassed the control group in median scores for comprehension, synthesis, and total scores, with a significant difference (p < 0.05). Additionally, most students expressed favorable opinions toward the interactive learning approach. CONCLUSIONS: It is recommended that studies on interactive learning be repeated in different subjects in nursing education. CLINICALTRIALS: gov ID:NCT05868278.

The evaluation of synchronous and asynchronous online learning: student experience, learning outcomes, and cognitive load.

BACKGROUND: The abrupt onset of the COVID-19 pandemic compelled universities to swiftly establish online teaching and learning environments that were not only immediately deployable but also conducive to high-quality education. This study aimed to compare the effectiveness of the online synchronous and asynchronous teaching formats in the dermatology lecture for undergraduate medical students, including academic performance, self-efficacy, and cognitive load. METHODS: A total of 170 fourth-year undergraduate medical students attending the dermatology lecture were included. The lecture was delivered using both the synchronous method (live online lecture via Webex meeting) and the asynchronous method (lecture videos shared on YouTube). The students had the freedom to choose their preferred method of attending the online lecture. The study assessed three main aspects: (1) learning outcomes measured through pretest, posttest, and retention test scores; (2) cognitive load experienced by students, including mental load and mental effort measured using eight items; and (3) satisfaction levels with each online teaching format. RESULTS: In this study, 70 students opted for the synchronous online lecture, while 100 students chose the asynchronous online lecture. Both synchronous and asynchronous teaching methods exhibited significant improvements in post and retention test scores compared to the pretest. Satisfaction levels, rated on a scale of 0-5, were generally high for both teaching methods, with no significant differences observed (4.6 for synchronous, 4.53 for asynchronous; p =.350). Regarding cognitive load, the synchronous method showed a significantly lower level than the asynchronous method (p =.0001). Subgroup analysis revealed no difference in mental effort (p =.0662), but the level of mental load was lower in the synchronous method (p =.0005). CONCLUSIONS: Both synchronous and asynchronous online teaching methods demonstrated improvements in learning outcomes and high levels of student satisfaction. However, the cognitive load experienced by students was lower in the synchronous setting compared to the asynchronous setting. These findings remind health professions educators that they would consider the students' cognitive load when designing online curricula.

Development and applications of the Anaesthetists' Non-Technical Skills behavioural marker system: a systematic review.

OBJECTIVES: To comprehensively synthesise evidence regarding the validity and reliability of the Anaesthetists' Non-Technical Skills (ANTS) behavioural marker system and its application as a tool for the training and assessment of non-technical skills to improve patient safety. DESIGN: Systematic review. DATA SOURCES: We employed a citation search strategy. The Scopus and Web of Science databases were searched for articles published from 2002 to May 2022. ELIGIBILITY CRITERIA: English-language publications that applied the ANTS system in a meaningful way, including its use to guide data collection, analysis and reporting. DATA EXTRACTION AND SYNTHESIS: Study screening, data extraction and quality assessment were performed by two independent reviewers. We appraised the quality of included studies using the Joanna Briggs Institute Critical Appraisal Checklists. A framework analysis approach was used to summarise and synthesise the included articles. RESULTS: 54 studies were identified. The ANTS system was applied across a wide variety of study objectives, settings and units of analysis. The methods used in these studies varied and included quantitative (n=42), mixed (n=8) and qualitative (n=4) approaches. Most studies (n=47) used the ANTS system to guide data collection. The most commonly reported reliability statistic was inter-rater reliability (n=35). Validity evidence was reported in 51 (94%) studies. The qualitative application outcomes of the ANTS system provided a reference for the analysis and generation of new theories across disciplines. CONCLUSION: Our results suggest that the ANTS system has been used in a wide range of studies. It is an effective tool for assessing non-technical skills. Investigating the methods by which the ANTS system can be evaluated and implemented for training within clinical environments is anticipated to significantly enhance ongoing enhancements in staff performance and patient safety. PROSPERO REGISTRATION NUMBER: CRD42022297773.

Alignment of learning objectives, assessments, and active learning to promote critical thinking in a first-year medical physiology course: lessons learned.

Medical students must be adept at critical thinking to successfully meet the learning objectives of their preclinical coursework. To encourage student success on assessments, the course director of a first-year medical physiology course emphasized the use of learning objectives that were explicitly aligned with formative assessments in class. The course director introduced the physiology discipline, learning objectives, and evidence-based methods of studying to students on the first day of class. Thereafter, class sessions started with a review of the learning objectives for that session and included active learning opportunities such as retrieval practice. The instructor provided short answer formative assessments aligned with the learning objectives, intended to help the students apply and integrate the concepts. Midsemester, students received a link to an online survey with questions on studying habits, class attendance, and student engagement. After finals, students were invited to participate in focus groups about their class experience. A qualitative researcher moderated focus groups, recorded responses, and analyzed the narrative data. Of 175 students, 95 submitted anonymous online surveys. Student engagement was significantly correlated with in-person class attendance (r = 0.26, T = 2.5, P = 0.01) and the completion of open-ended formative assessments (r = 0.33, T = 3.3, P = 0.001). Focus groups were held via videoconference. From the class, 14 students participated in 4 focus groups; focus group participants were mostly women (11 of 14) and mostly in-class attendees (13 of 14). The students in this sample valued critical thinking but misunderstood expectations on exams and few students used learning objectives to study.NEW & NOTEWORTHY We introduced formative assessments and study techniques to first-year medical students in a physiology course. Mastery of learning objectives was emphasized as the key to success. We asked how they studied physiology through an anonymous online survey and focus group interviews. The students enjoyed physiology but had difficulty with exam expectations. Helping students use learning objectives to guide their study may lead to improved exam scores. It may also help administrators meet their curriculum goals.

Relationship of emotional intelligence and capability of answering higher-order knowledge questions in physiology among first-year medical students.

Emotional intelligence (EI) has a positive correlation with the academic performance of medical students. However, why there is a positive correlation needs further exploration. We hypothesized that the capability of answering higher-order knowledge questions (HOQs) is higher in students with higher EI. Hence, we assessed the correlation between EI and the capability of medical students to answer HOQs in physiology. First-year undergraduate medical students (n = 124) from an Indian medical college were recruited as a convenient sample. EI was assessed by the Schutte Self-Report Emotional Intelligence Test (SSEIT), a 33-item self-administered validated questionnaire. A specially designed objective examination with 15 lower-order and 15 higher-order multiple-choice questions was conducted. The correlation between the examination score and the EI score was tested by Pearson's correlation coefficient. Data from 92 students (33 females and 59 males) with a mean age of 20.14 ± 1.87 yr were analyzed. Overall, students got a percentage of 53.37 ± 14.07 in the examination, with 24.46 ± 9.1 in HOQs and 28.91 ± 6.58 in lower-order knowledge questions (LOQs). They had a mean score of 109.58 ± 46.2 in SSEIT. The correlation coefficient of SSEIT score with total marks was r = 0.29 (P = 0.0037), with HOQs was r = 0.41 (P < 0.0001), and with LOQs was r = 0.14 (P = 0.19). Hence, there is a positive correlation between EI and the capability of medical students to answer HOQs in physiology. This study may be the foundation for further exploration of the capability of answering HOQs in other subjects.NEW & NOTEWORTHY This study assessed the correlation between emotional intelligence (EI) and the capability of medical students to answer higher-order knowledge questions (HOQs) in the specific context of physiology. The finding reveals one of the multifaceted dimensions of the relationship between EI and academic performance. This novel perspective opens the door to further investigations to explore the relationship in other subjects and other dimensions to understand why students with higher EI have higher academic performance.

Using Concurrent Complication Reporting to Evaluate Resident Critical Thinking and Enhance Adult Learning.

OBJECTIVE: Critical thinking and accurate case analysis is difficult to quantify even within the context of routine morbidity and mortality reporting. We designed and implemented a HIPAA-compliant adverse outcome reporting system that collects weekly resident assessments of clinical care across multiple domains (case summary, complications, error analysis, Clavien-Dindo Harm, cognitive bias, standard of care, and ACGME core competencies). We hypothesized that incorporation of this system into the residency program's core curriculum would allow for identification of areas of cognitive weakness or strength and provide a longitudinal evaluation of critical thinking development. DESIGN: A validated, password-protected electronic platform linked to our electronic medical record was used to collect cases weekly in which surgical adverse events occurred. General surgery residents critiqued 1932 cases over a 4-year period from 3 major medical centers within our system. These data were reviewed by teaching faculty, corrected for accuracy and graded utilizing the software's critique algorithm. Grades were emailed to the residents at the time of the review, collected prospectively, stratified, and analyzed by post-graduate year (PGY). Evaluation of the resident scores for each domain and the resultant composite scores allowed for comparison of critical thinking skills across post-graduate year (PGY) over time. SETTING: Data was collected from 3 independently ACGME-accredited surgery residency programs over 3 tertiary hospitals within our health system. PARTICIPANTS: General surgery residents in clinical PGY 1-5. RESULTS: Residents scored highest in properly identifying ACGME core competencies and determining Clavien-Dindo scores (p < 0.006) with no improvement in providing accurate and concise clinical summaries. However, residents improved in recording data sufficient to identify error (p < 0.00001). A positive linear trend in median scores for all remaining domains except for cognitive bias was demonstrated (p < 0.001). Senior residents scored significantly higher than junior residents in all domains. Scores > 90% were never achieved. CONCLUSIONS: The use of an electronic standardized critique algorithm in the evaluation and assessment of adverse surgical case outcomes enabled the measure of residents' critical thinking skills. Feedback in the form of teaching faculty-facilitated discussion and emailed grades enhanced adult learning with a steady improvement in performance over PGY. Although residents improved with PGY, the data suggest that further improvement in all categories is possible. Implementing this standardized critique algorithm across PGY allows for evaluation of areas of individual resident weakness vs. strength, progression over time, and comparisons to peers. These data suggest that routine complication reporting may be enhanced as a critical thinking assessment tool and that improvement in critical thinking can be quantified. Incorporation of this platform into M&M conference has the potential to augment executive function and professional identity development.

Assessing the performance of Bing Chat artificial intelligence: Dental exams, clinical guidelines, and patients' frequent questions.

OBJECTIVES: Bing Chat is a large language model artificial intelligence (AI) with online search and text generating capabilities. This study assessed its performance within the scope of dentistry in: (a) tackling exam questions for dental students, (ii) providing guidelines for dental practitioners, and (iii) answering patients' frequently asked questions. We discuss the potential of clinical tutoring, common patient communication and impact on academia. METHODS: With the aim of assessing AI's performance in dental exams, Bing Chat was presented with 532 multiple-choice questions and awarded scores based on its answers. In evaluating guidelines for clinicians, a further set of 15 questions, each with 2 follow-up questions on clinical protocols, was presented to the AI. The answers were assessed by 4 reviewers using electronic visual analog scale. In evaluating answers to patients' frequently asked questions, another list of 15 common questions was included in the session, with respective outputs assessed. RESULTS: Bing Chat correctly answered 383 out of 532 multiple-choice questions in dental exam part, achieving a score of 71.99 %. As for outlining clinical protocols for practitioners, the overall assessment score was 81.05 %. In answering patients' frequently asked questions, Bing Chat achieved an overall mean score of 83.8 %. The assessments demonstrated low inter-rater reliability. CONCLUSIONS: The overall performance of Bing Chat was above the regularly adopted passing scores, particularly in answering patient's frequently asked questions. The generated content may have biased sources. These results suggest the importance of raising clinicians' awareness of AI's benefits and risks, as well as timely adaptations of dental education curricula, and safeguarding its use in dentistry and healthcare in general. CLINICAL SIGNIFICANCE: Bing Chat AI performed above the passing threshold in three categories, and thus demonstrated potential for educational assistance, clinical tutoring, and answering patients' questions. We recommend popularizing its benefits and risks among students and clinicians, while maintaining awareness of possible false information.

Developing and validating the Unhurried Conversations Assessment Tool (UCAT).

OBJECTIVE: Given the importance of unhurried conversations for providing careful and kind care, we sought to create, test, and validate the Unhurried Conversations Assessment Tool (UCAT) for assessing the unhurriedness of patient-clinician consultations. METHODS: In the first two phases, the unhurried conversation dimensions were identified and transformed into an assessment tool. In the third phase, two independent raters used UCAT to evaluate the unhurriedness of 100 randomly selected consultations from 184 videos recorded for a large research trial. UCAT's psychometric properties were evaluated using this data. RESULTS: UCAT demonstrates content validity based on the literature and expert review. EFA and reliability analyses confirm its construct validity and internal consistency. The seven formative dimensions account for 89.93% of the variance in unhurriedness, each displaying excellent internal consistency (α > 0.90). Inter-rater agreement for the overall assessment item was fair (ICC = 0.59), with individual dimension ICCs ranging from 0.26 (poor) to 0.95 (excellent). CONCLUSION: UCAT components comprehensively assess the unhurriedness of consultations. The tool exhibits content and construct validity and can be used reliably. PRACTICE IMPLICATIONS: UCAT's design and psychometric properties make it a practical and efficient tool. Clinicians can use it for self-evaluations and training to foster unhurried conversations.